Papers with natural language processing applications
Bias and Fairness in Natural Language Processing (D19-2)
Copied to clipboard
| Challenge: | a tutorial will review the history of bias and fairness studies in machine learning and language processing . |
| Approach: | This tutorial reviews the history of bias and fairness studies in machine learning and language processing . it presents recent community effort to quantify and mitigat bias in natural language processing models . |
| Outcome: | This tutorial reviews the history of bias and fairness studies in machine learning and language processing . it aims to quantify and mitigate bias in natural language processing models for a wide spectrum of tasks . |
Bad Form: Comparing Context-Based and Form-Based Few-Shot Learning in Distributional Semantic Models (D19-61)
Copied to clipboard
| Challenge: | Word embeddings are an essential component of many natural language processing applications. |
| Approach: | They propose 3 new tasks to obtain higher-quality vectors for word embeddings . they use word forms in training data that are related to word forms themselves . |
| Outcome: | The proposed methods improve the performance of both baseline and advanced models on 4 out of 6 tasks. |
Repulsive Attention: Rethinking Multi-head Attention as Bayesian Inference (2020.emnlp-main)
Copied to clipboard
Bang An, Jie Lyu, Zhenyi Wang, Chunyuan Li, Changwei Hu, Fei Tan, Ruiyi Zhang, Yifan Hu, Changyou Chen
| Challenge: | Existing studies show that multi-head attention is an effective module in deep neural networks, but there are no explicit mechanisms guaranteeing this property. |
| Approach: | They propose a non-parametric approach that explicitly improves the repulsiveness in multi-head attention and consequently strengthens model’s expressiveness. |
| Outcome: | The proposed approach improves the repulsiveness in multi-head attention and strengthens model’s expressiveness. |
SwahBERT: Language Model of Swahili (2022.naacl-main)
Copied to clipboard
| Challenge: | Social media and Internet forums are valuable sources of citizens’ opinions, which can be analyzed for community development and user behavior analysis. |
| Approach: | They present a pre-training and annotated datasets of Swahili and an emotion classification datasets that are manually annotating by two native Swahils. |
| Outcome: | The proposed model outperforms existing monolingual language model in almost all downstream tasks. |
Multi-Source Text Classification for Multilingual Sentence Encoder with Machine Translation (2024.naacl-srw)
Copied to clipboard
| Challenge: | Pre-trained multilingual sentence encoders suffer from performance degradation for non-English languages. |
| Approach: | They propose a method of machine translating a source sentence into English and then inputting it together with the source sentence in a multi-source manner. |
| Outcome: | The proposed method improves the performance of pre-trained multilingual sentence encoders in Japanese on sentiment analysis and topic classification tasks. |
Adversarial Feature Adaptation for Cross-lingual Relation Classification (C18-1)
Copied to clipboard
| Challenge: | Existing methods for relation classification exploit monolingual data due to lack of annotated data in other languages. |
| Approach: | They propose an adversarial feature adaptation approach for cross-lingual relation classification using a generative adversarial network. |
| Outcome: | The proposed approach yields an improvement of 5.7% over the state-of-the-art. |
MELOV: Multimodal Entity Linking with Optimized Visual Features in Latent Space (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to multimodal entity linking focus on textual contexts but lack in social media vision modality. |
| Approach: | They propose a latent space vision feature optimization framework MELOV to address these challenges . they exploit variational autoencoder to mine shared information and generate text-based visual features . |
| Outcome: | The proposed framework is superior to existing methods on three benchmark datasets. |
Cross-lingual and Cross-domain Transfer Learning for Automatic Term Extraction from Low Resource Data (2022.lrec-1)
Copied to clipboard
| Challenge: | Automatic Term Extraction (ATE) is a key component for domain knowledge understanding and can be used for further NLP applications. |
| Approach: | They propose to fine-tune pre-trained BERT models for automatic Term Extraction (ATE) using cross-lingual and cross-domain transfer learning to extract single and multi-word terms. |
| Outcome: | The proposed models can capture cross-domain and cross-lingual terminologically-marked contexts shared by terms, opening a new design-pattern for ATE. |
Few-shot Learning for Slot Tagging with Attentive Relational Network (2021.eacl-main)
Copied to clipboard
| Challenge: | Recent studies have used metric-based learning in computer vision but not slot tagging. |
| Approach: | They propose a metric-based learning architecture that extends relation networks by leveraging pretrained contextual embeddings such as ELMO and BERT and by using attention mechanism. |
| Outcome: | The proposed method outperforms state-of-the-art methods on SNIPS data on a slot tagging task with a large amount of hand-labeled data. |
Urdu Word Embeddings (L18-1)
Copied to clipboard
| Challenge: | Recent advances in distributional semantics have led to the rise of neural network-based models that use unsupervised learning to represent words as dense, distributed vectors, called 'word embeddings' embedders hold key to improving natural language processing for low-resource languages, since they require significant time and manpower. |
| Approach: | They train a skip-gram model on 140 million Urdu words to create the first large-scale word embeddings for the Urdu language. |
| Outcome: | The proposed models capture high degree of syntactic and semantic similarity between words and are able to generalize well on the Urdu translation task. |
Bilingual Lexicon Induction for Low-Resource Languages using Graph Matching via Optimal Transport (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing literature on bilingual lexicon induction fails in low-resource scenarios . a language dataset is considered low- resource based on its own embedding space . |
| Approach: | They propose a graph-matching method that improves bilingual lexicon induction performance across 40 language pairs using optimal transport. |
| Outcome: | The proposed method is especially strong with low amounts of supervision. |
Few-shot Knowledge Graph Relational Reasoning via Subgraph Adaptation (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing methods to predict unseen triplets from knowledge graphs are limited by spurious information in KGs. |
| Approach: | They propose a framework that adapts contextualized graphs to subgraphs generated from support and query triplets to perform the prediction. |
| Outcome: | The proposed framework extracts more comprehensive information from support triplets while minimizing spurious information when predicting query triplet. |
Recurrent Neural Networks as Weighted Language Recognizers (N18-1)
Copied to clipboard
| Challenge: | Recent experiments show that RNNs outperform other methods in assigning high probability to held-out English text. |
| Approach: | They focus on the single-layer, ReLU-activation, rational-weight RNNs with softmax . they show that most problems for such RNN are undecidable . |
| Outcome: | The proposed model outperforms other methods in assigning high probability to held-out English text. |
One-Shot Relational Learning for Knowledge Graphs (D18-1)
Copied to clipboard
| Challenge: | Existing studies on knowledge graph completion require a large number of positive examples for each relation, but long-tail relations are more common in KGs and those newly added relations do not have many known triples for training. |
| Approach: | They propose a one-shot relational learning framework that utilizes the knowledge distilled by embedding models and learns a matching metric by considering both the learned embeddments and one-hop graph structures. |
| Outcome: | The proposed framework improves on existing embedding models and eliminates the need for retraining when dealing with newly added relations. |
Cold-Start and Interpretability: Turning Regular Expressions into Trainable Recurrent Neural Networks (2020.emnlp-main)
Copied to clipboard
| Challenge: | Neural networks typically need large labeled data for training and are not easily interpretable. |
| Approach: | They propose a type of recurrent neural networks that combine neural networks and regular expression rules. |
| Outcome: | The proposed recurrent neural networks outperform previous neural approaches in low- and zero-shot scenarios and remain very competitive in rich-resource settings. |
When Beards Start Shaving Men: A Subject-object Resolution Test Suite for Morpho-syntactic and Semantic Model Introspection (2020.coling-main)
Copied to clipboard
| Challenge: | In German, subject-object resolution is the second most error-prone task in syntactic parsing, after prepositional phrase attachment. |
| Approach: | They introduce the SORTS Subject-Object Resolution Test Suite of German minimal sentence pairs for model introspection. |
| Outcome: | The proposed test suite is based on the annotations of 8,502 transitive clauses with 8 word order patterns, 5 morphological and syntactic and 11 semantic property classes. |
SEAG: Structure-Aware Event Causality Generation (2023.findings-acl)
Copied to clipboard
Zhengwei Tao, Zhi Jin, Xiaoying Bai, Haiyan Zhao, Chengfeng Dou, Yongqiang Zhao, Fang Wang, Chongyang Tao
| Challenge: | Current methods for extracting event causality are limited by the lack of cross-task dependencies and may cause error propagation. |
| Approach: | They propose an approach for Structure-Aware Event Causality Generation (SEAG) they generate the ECG structure using a pre-trained language model and perform structural discriminative training alongside auto-regressive generation. |
| Outcome: | The proposed method is effective in extracting event causality from text. |
Pushing the Limits of AMR Parsing with Self-Learning (2020.findings-emnlp)
Copied to clipboard
Young-Suk Lee, Ramón Fernandez Astudillo, Tahira Naseem, Revanth Gangi Reddy, Radu Florian, Salim Roukos
| Challenge: | Abstract Meaning Representation (AMR) parsing has experienced a notable growth in performance in the last two years due to the impact of transfer learning and the development of novel architectures specific to AMR. |
| Approach: | They propose to use AMR annotations to generate synthetic text and refine actions oracle without additional human annotations for AMR parsing. |
| Outcome: | The proposed models improve on AMR 1.0 and 2.0 without human annotations. |
Revealing the Numeracy Gap: An Empirical Investigation of Text Embedding Models (2026.findings-eacl)
Copied to clipboard
| Challenge: | Text embedding models are widely used in natural language processing but are often benchmarked on tasks that do not require understanding nuanced numerical information in text. |
| Approach: | They evaluate 13 widely used text embedding models and find they struggle to capture numerical details accurately. |
| Outcome: | The proposed models struggle to capture nuanced numerical details accurately, despite being benchmarked on tasks that do not require understanding nuance. |
Fine- and Coarse-Granularity Hybrid Self-Attention for Efficient BERT (2022.acl-long)
Copied to clipboard
| Challenge: | Transformer-based pre-trained models achieve state-of-the-art results, but they can be prohibitively costly. |
| Approach: | They propose a fine- and coarse-granularity hybrid self-attention that shortens the computational sequence length in self- attention by progressively shortening the computational time. |
| Outcome: | The proposed model reduces computation cost by shortening the computational sequence length in self-attention. |
Textomics: A Dataset for Genomics Data Summary Generation (2022.acl-long)
Copied to clipboard
| Challenge: | Summarizing biomedical discovery from genomics data is done manually but is slowing down the progress of scientific discovery. |
| Approach: | They propose a novel task of generating sentences to summarize a genomics data matrix using neural text generation. |
| Outcome: | The proposed model improves on the previous models and can be applied to other biomedical and natural language processing applications. |
Breeding Gender-aware Direct Speech Translation Systems (2020.coling-main)
Copied to clipboard
| Challenge: | In automatic speech translation, traditional cascade approaches involving separate transcription and translation steps are giving ground to more robust direct solutions. |
| Approach: | They compare different approaches to inform direct ST models about the speaker’s gender and test their ability to handle gender translation from English into Italian and French. |
| Outcome: | The proposed models outperform strong but gender-unaware direct ST models in the translation of English into Italian and French. |
Curation of Benchmark Templates for Measuring Gender Bias in Named Entity Recognition Models (2024.lrec-main)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) models are susceptible to gender bias . benchmark datasets are curated specifically for a given NLP task . |
| Approach: | They propose to filter out benchmark templates with a higher probability of detecting gender bias in NER models. |
| Outcome: | The proposed method is based on masked token prediction and tested in English and german using the corresponding fine-tuned BERT base model. |
LIdioms: A Multilingual Linked Idioms Data Set (L18-1)
Copied to clipboard
| Challenge: | Recent studies have focused on linguistic data sets that are bilingual on the Linguistic Linked Open Data (LLOD) 1 . |
| Approach: | They describe a multilingual RDF representation of idioms currently containing five languages . they use a model to structure the data and a method to link the data to well-known multilingual data sets such as BabelNet. |
| Outcome: | The proposed model complies with best practices according to Linguistic Linked Open Data Community. |
SpanPredict: Extraction of Predictive Document Spans with Neural Attention (2021.naacl-main)
Copied to clipboard
| Challenge: | identifying predictive text in clinical notes can be as important as the predictions themselves . identifying specific content in clinical note descriptions may illuminate previously unknown risk factors . |
| Approach: | They propose a method for identifying predictive text in clinical notes . they use linear attention to formalize the problem as predictive extraction . |
| Outcome: | The proposed model preserves differentiability and allows scalable inference via stochastic gradient descent. |
NMT and PBSMT Error Analyses in English to Brazilian Portuguese Automatic Translations (2020.lrec-1)
Copied to clipboard
| Challenge: | Recent work proposes neural machine translation (NMT) for Brazilian Portuguese. |
| Approach: | They propose a neural machine translation approach that generates equivalent sentences in target language and source language. |
| Outcome: | The proposed approach outperforms phrase-based statistical machine translation systems for some pairs of languages. |
Multiword Expression aware Neural Machine Translation (2020.lrec-1)
Copied to clipboard
| Challenge: | Multiword Expressions (MWEs) are a pervasive phenomenon in all natural languages and challenge NLP applications because of their unpredictable morpho-syntactic and lexico--semantic behaviour. |
| Approach: | They propose to use linguistic resources to improve MWE translation and MWE generation by up to 5.09 BLEU points on MWE test sets. |
| Outcome: | The proposed annotation and data augmentation improve translation quality and increase performance by up to 5.09 BLEU points on MWE test sets. |
Morfessor EM+Prune: Improved Subword Segmentation with Expectation Maximization and Pruning (2020.lrec-1)
Copied to clipboard
| Challenge: | Subword segmentation is a standard preprocessing step in many neural approaches to natural language processing. |
| Approach: | They propose to train a unigram subword model using a recursive algorithm and lexicon pruning algorithm. |
| Outcome: | The proposed method improves on the original training algorithm and improves morphological segmentation accuracy. |
A Spelling Correction Corpus for Multiple Arabic Dialects (2020.lrec-1)
Copied to clipboard
| Challenge: | Arabic dialects are non-standard varieties of Arabic commonly spoken across the Arab world, but lack standard orthographies. |
| Approach: | They present a corpus of 10,000 sentences from five Arabic city dialects represented in the Conventional Orthography for Dialectal Arabic (CODA) they use a bootstrapping technique to speed up annotation and compare similarity between dialects before and after CODA annotation. |
| Outcome: | The proposed method speeds up the annotation process and shows similarity between the dialects before and after CODA annotation. |
NNE: A Dataset for Nested Named Entity Recognition in English Newswire (P19-1)
Copied to clipboard
| Challenge: | Named entity recognition (NER) is widely used in downstream tasks but most tools focus on flat mention structure over coarse schemas. |
| Approach: | They describe a fine-grained, nested named entity dataset over the Wall Street Journal portion of the Penn Treebank. |
| Outcome: | The proposed dataset comprises 279,795 mentions of 114 entity types with up to 6 layers of nesting. |
EVIT: Event-Oriented Instruction Tuning for Event Reasoning (2024.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have made significant advances in event reasoning . however, smaller instruction-tuned models do not consistently demonstrate exceptional proficiency . |
| Approach: | They propose an event-oriented instruction tuning technique to train a large language model . they propose a structure named event quadruple which contains the structure and semantics of events . |
| Outcome: | The proposed model achieves competitive performances on event reasoning tasks. |
Evaluating the Morphosyntactic Well-formedness of Generated Texts (2021.emnlp-main)
Copied to clipboard
Adithya Pratapa, Antonios Anastasopoulos, Shruti Rijhwani, Aditi Chaudhary, David R. Mortensen, Graham Neubig, Yulia Tsvetkov
| Challenge: | Text generation systems are ubiquitous in natural language processing applications, but evaluation of these systems remains a challenge, especially in multilingual settings. |
| Approach: | They propose a metric to evaluate the morphosyntactic well-formedness of text using its dependency parse and morphologically-rich rules of the language. |
| Outcome: | The proposed metric can evaluate the morphosyntactic well-formedness of text using its dependency parse and morphologically-rich rules of the language. |
Minimally Supervised Learning of Affective Events Using Discourse Relations (D19-1)
Copied to clipboard
| Challenge: | Existing methods for learning affective events that trigger positive or negative sentiment are difficult because of the unbounded combinatorial nature of language. |
| Approach: | They propose to propagate affective polarity using discourse relations using a small seed lexicon and large raw corpus. |
| Outcome: | The proposed method learns affective events effectively without manually labeled data, and improves supervised learning when labeles are small. |
Few-shot Temporal Pruning Accelerates Diffusion Models for Text Generation (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing acceleration methods for text generation ignore the importance of the distribution of sampling steps, resulting in slow sampling rates. |
| Approach: | They propose a technique to accelerate diffusion models for text generation without additional training by using a Bayesian optimization approach. |
| Outcome: | The proposed technique achieves 400x acceleration even with minimal sampling steps after down to less than 1 minute of optimization yielding a competitive performance even with minimum sampling steps. |
A Large-Scale Japanese Dataset for Aspect-based Sentiment Analysis (2022.lrec-1)
Copied to clipboard
| Challenge: | Aspect-based sentiment analysis (ABSA) has not been explored in the Japanese language . there is no standard Japanese dataset available for ABSA task in the language - a paper by cnn. |
| Approach: | They propose to use a Japanese aspect-based sentiment analysis dataset for hotel reviews domain . they propose to include 53,192 review sentences with seven aspect categories and two polarity labels . |
| Outcome: | The proposed dataset contains 53,192 review sentences with seven aspect categories and two polarity labels. |
The Art of Defending: A Systematic Evaluation and Analysis of LLM Defense Strategies on Safety and Over-Defensiveness (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent work on Large Language Models (LLMs) has identified a number of approaches to protect against their vulnerabilities and safety. |
| Approach: | They propose to use a large collection of safe and unsafe prompts to evaluate various LLM defense strategies over both ‘safety’ and ‘over-defensiveness’. |
| Outcome: | The proposed defense strategies are compared on multiple state-of-the-art LLMs and show that they are effective against both ‘safety’ and ‘over-defensiveness’. |
Spanish Resource Grammar Version 2023 (2024.lrec-main)
Copied to clipboard
| Challenge: | Using the Freeling morphological analyzer, we encode a strict notion of grammaticality in the Spanish resource grammar. |
| Approach: | They propose to use the HPSG formalism to encode a Spanish resource grammar with a manually verified treebank of 2,291 sentences. |
| Outcome: | The proposed grammars encode a complex set of hypotheses about syntax and a strict notion of grammaticality making them a resource for natural language processing applications in computer-assisted language learning. |
Unmasking Biases: Exploring Gender Bias in English-Catalan Machine Translation through Tokenization Analysis and Novel Dataset (2024.lrec-main)
Copied to clipboard
| Challenge: | a new dataset focuses on gender-neutral terms that necessitate gendered translations in Catalan. |
| Approach: | They propose to use a new dataset to evaluate gender bias in machine translation . they train four MT systems using different tokenization techniques . |
| Outcome: | The proposed dataset focuses on gender-neutral terms necessitating gendered translations in Catalan. |